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INTRODUCTION 

0.1 Scientific Structure of the Artificial Intelligence Laboratory 

The activities of the Artificial Intelligence Laboratory can be 
viewed under three main aspects: 

1. Artificial Intelligence: Understanding the principles of 
making intelligent machines along the lines discussed in 
previous proposals, and elaborated below. 

2. Natural Intelligence: As we understand intelligence better 
we see fewer differences between the problems of under- 
standing human and machine intelligence. 

We have been increasingly able to translate our Ideas about 
programming machines into ideas about educating children, and are 
currently developing systematic methods in elementary education. And 
conversely, we attribute to our observations and experience in the 
latter activities much of what we believe are Important new conceptions 
of how to organize knowledge for programs that really understand. 

3. Mathematical Theories: This aspect 1s relevant not only 
because we often need to solve specific mathematical 
problems but especially because we are firmly committed 
to maintaining a mathematical style In the laboratory. 
In many centers we have seen decline and deterioration 
following an apparently successful "experiment" In 
artificial intelligence because the principles behind 
the performance were not understood, hence the limita- 
tions unseen. 
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0,2 Organization of the Laboratory 

Organizationally our laboratory is divided into a number of 
groups with a certain degree of overlap (both 1n people and 1n Interest}, 

VISION: programing machines to see 

ROBOTICS: programming machanical manipulations 

LANGUAGE: programs that understand English 

PLANNER: implementation of a new programming language 

PROGRAM-UNDERSTANDING: programs that understand processes 

MATHEMATICS: especially schemata, complexity, theory of 

computation 
MATHUB: (in cooperation with the group In Project MAC) 

■I.T.5. 11 : The A.I. Lab's time-shared computer system 

DEVaOPMENTAL PSYCHOLOGY: the educational research group (NSF) 

HARDWARE: experimental shop facility 

ADMINISTRATION. 

For the past year, the Artificial Intelligence Laboratory has 
been an Independent M.I.T. Laboratory, separate from Project MAC, 
(The group was started in 1958 as part of the Research Laboratory of 
Electronics and the M.I.T. Computation Center and later became a group 
of Project MAC. It has grown to a size and complexity that now needs 
its own administration.) The funding Is mainly from ARPA; the studies 
in Education and Child Development are supported by the NSF, and there 
are some smaller support contributions from NASA, NIH, and from some 
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sources of fellowships and assistantships. 

In the area of ROBOTICS, we have come to the conclusion that 
the research directions that have been followed, while extremely 
productive for Artificial Intelligence research, have not been effective 
enough In stimulating practical developments 1n the area of advanced 
automation. Accordingly, we have proposed to ARPA, 1n a separate 
supplementary letter, to consolidate what we now know 1n the design 
of a complete and easily copied mini -robotics laboratory system, as a 
packaged system, so that other centers can get started on applied 
research in that area. 
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1, Progress In the Hand-Eye Project 

Progress in our laboratory can be seen on different levels. 
Very specific projects, such as the hand-eye work supported by ARPA t 
achieve very specific and increasingly ambitious goals* Several years 
ago we were struggling with the problem of making a computer see any- 
thing at all * and were proud of the first demonstration In which the 
computer could see a visually isolated, clear, well-Illuminated 
shadow-free cube well enough to add It to a tower. Last year our 
seeing machine was able to look at a coirplex of mutually obscuring 
objects, analyze the scene visually, dismantle it and put it together 
again in mirror image. 

The first scene copied was as shown below. Note 

that objects obscure one another by way of support 
and in-front-of relationships. The objects are not 
of any particular size; both size and position are 
determined in the copying effort as required. 




Since then we have increased its ability to tolerate less than perfect 
lighting conditions, see shadows, deal with even more obscured objects 
and so on. 

But, our progress appears as quantitatively continuous only 
when viewed very superficially. A deeper examination shows qualitative 
changes in the methodology of the work, in the kind of question we ask 
and answer, and above all in the kind of theoretical conclusions we 
draw from it. To define the trend of our work and, particularly, to 
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place next year's work in its proper context, we need to review our 
progress on these more fundamental levels. 

So, let us review briefly the progress in vision, this time 
examining the style cf operation of the programs rather than their 
outer behavior. 

Seeing and picking up an Isolated object under good conditions 
does not actually need any of the techniques of programming or 
theoretical concepts we see as characteristic of "artificial intelligence 1 
It can be handled as an interesting, difficult, but typical problem 
in classical programing and engineering. 

The main initial direction forward from the simple case 1n 
our laboratory was the development of what we now call "mini-theories" 
of In^ortant visual phenomena. Some of these concerned problems of an 
optical character> such as the light distribution at the edge of an 
object, the deduction of surface curvature from shading, the local 
structure of shadow lines and so on. Others were of a geometric 
character, such as the classification of vertices and edges Into 
regions, the dissection of scenes into separate objects, defining 
relations between objects in three dimensional space. 

We consider our programs as belonging to Artificial Intelligence 
from the time they begin to use such mini-theories* But merely using 
them is only a beginning; the truly deep questions are connected 
with how they are used. And the major focus of our work has turned 
toward defining and understanding phenomena of interaction of diverse 
kinds of knowledge. 
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An early example of such interaction in the vision 
system occurred 1n the process of locating objects 
In space. One straightforward method using focus 
was used with a more complicated module that depends 
on heuristlcally derived conjectures about how objects 
in the scene support each other. The focus method 
Is less accurate, but is inmune to the gross blunders 
occasionally made by the support-dependent module. 
When used together one can get both great reliability 
and great precision. 

The simplest way to use mini -theories such as those listed above 
is separately , for exanple in a pass-oriented or hierarchical program 
structure where "optic" mini-theories might be used for line detection, 
geometric mini-theories for parsing scenes and so on. There is no 
doubt that useful results can be obtained this way and we have demon- 
strated some; but 1t 1s now clear that the real pay-off is 1n a different, 
ultimately more powerful direction, illustrated by the idea of heter- 
arcMcal program structure. 

A fair image of what we mean by this can be obtained by thinking 
of ways in which human specialists might be integrated into a large 
organization. The pass-oriented model uses the following pattern for 
solving a problem: the problem is divided in advance into stages 
or sub-problems, which are passed out to the specialists; each special- 
ist sees his part of the problem* does what he can with it, passes 
on the result and is through. Heterarchical organization implies the 
possibility of conrnuni cation between specialists 1 consultations, 
sending the problem back for further study and so on. 

The advantages and disadvantages of each mode of organization 
are plain enough: the first structure lends itself to orderly work 
but suffers from rigidity and if there ts any flaw In the original 
plan it will fail at that point. The second structure is much more 
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powerful on condition that the process does not fall into chaotic 
disorder . Experience of human organizations shows that a free inter- 
action can work only if somewhere in the background is an infra- 
structure of conventions and understandings. This is equally true 
of program organization: one of our central technical problems is 
how to set up such organizational infra-structures. 

During the past year it has become clear that our conceptual 
and technical tools for this task have reached a critical mass, A 
new style of programming and thinking about programs for Artificial 
Intelligence has almost explosively spread through our own laboratory 
and has even already begun to affect the work at other centers of 
research. The first major example of programing In the new style 

was T- Winograd's program discussed 1n last year's proposal to ARPA. 
Since then, as we anticipated in that proposal » our Vision System 
has been recast into the new form and several new projects have 
adopted it from the outset. The following section contains a summary 
statement of the main features of these new programs. A more detailed 
account can be found in our forthcoming Progress Report (available 

1971), which has taken the form of a monograph on the funda- 
mental problems of Artificial Intelligence (and is therefore a piece 
of progress in its own right as well as a report on this and other 
progress). Me do want to emphasize our belief that what we are doing 
really is quite different from the approach of Artificial Intelligence 
generally followed up to now in other centers and expressed in the 
recent round of books on the subject. We see the difference as funda- 
mentally this: faced with the apparent diversity of kinds of knowledge, 
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the common approach at other centers 1s to seek ways to render It 
more uniform so that very general, "logically clear" methods can be 
used; our approach is to accept the diversity of knowledge as real 
and inevitable, and find ways to manage diversity rather than eliminate 
It. 
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2. Knowledge And Common Sense 

An exciting aspect of the new programming style is that it 
promises at last to break up the stereotype of the computer's slavish 
dullness, or "super-speed moron" character. A few years ago, the remarks 
of the previous section would have been appropriate to describing the 
differences between men and computer programs; now they contrast the 
new with the old programs! A good example of a degree of "common-sense" 
is supplied by the interactions of the programs in the "Blocks World" 
system that is responsible for the behavior 1n the attached dialog — 
(Appendix). Pie syntactic and semantic systems generate construction 
goals, like "Build a steeple." The definition of the goal ("a steeple 
Is a stack which contains two cubes and a pyramid 11 ) is converted Into 
a plan — a step-by-step specification of subgoals -- that is interpreted 
as a program for building a steeple; find a cube; put another cube 
on it, put a pyramid on that. Now when a first cube 1s found, 
Its top might be cluttered with other objects. We do not want the 
robot — like the assembly-machine 1n Chaplin's Modern Times -- to 
smash the second cube down willy-nilly. But we don't want, either, 
to have to write Into our plan: "find a first cube and remove the 
things on it" because (1} 1t is so obvious to say It and (2) it only 
sweeps the problem under the njg: the program will have to find a 
clear surface somewhere to put down the Junk it takes off the first 

cube. 

Wlnograd's system faces the problem once and for all. An 
almost autonomous network of statements and procedures "know" that 
to put one thing on another there must be a place it will fit. If 
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there 1s no such place one must make one (or fall). To make a place 
one must move something else, and (recursively) put ft somewhere else 
where it^wlll fit, etc. But one cannot put something on Itself , even 
If there is room, because it won't be where it was at the future moment 
of setting 1t down after moving 1t, etc- etc- Now, instead of writing 
this sort of thing into each particular application program, we create 
once and for all a "micro-world* of knowledge about how things are 
supported, how supports change when things are moved, and so forth. 
This knowledge fs invoked by the occurrence of patterns either in the 
outer world, or in goal -statements, whenever they occur, and the 
conmon-sense processes intervene and take over the actions until their 
invoking patterns disappear. For example: there 1s in the micro-world 
a statement whose effect is 

"If A 1s supported by B, and A 1s moved, then erase 
from the current description of the situation any 
statement of the form 'B supports A. ,M 
Notice the indirect character of this. It is a statement not about 
the physics of support but about when to forget statements concerning 
supportl (When you paint an object, and then move 1t, analogous 
statements should not disappear.) One could conceivably do without 
this advice, at the cost of recomputing after each change 1n the world 
all the relations between all subsets of objects. This 1s Impractical 
and is a common cause of examples in which a system works on "toy 
problems" and collapses on real problems. Or one could do without this 
advice, at the cost of recomputing after each change in the world all 
the logical consequences of that change; this leads 1n a different way 
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to collapse. One car make heuristic compromises: motions change 
geometrical relations but not (usually) other attributes of objects, 
The art — and science, eventually — here Is in finding what are 
the points that are so immediate that one should know them directly, 
and which can be left to more general but more laborious deductive 
systems. 

Connor sense is not magi j. If we want our computer to act as 
though it knows the elementary strategies about physics and geometry, 
we must give it that knowledge somehow. But we need not do this anew 
for each program! So our goal is to learn how to refine the Ideas 
in the Blocks World, and the ideas in pattern-matching invocations that 
make this knowledge engage relevant situations, so that we can keep 
this "data" permanently in the system. Then any program written therein 
will automatically behave sensibly In that sphere of activity. 
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3, Sore Features of the New Programing Style 

(a) Use of the language PLANNER 

Programming languages such as ALGOL are designed to make 
It easy to express the kind of statement certain classes 
of user*s are likely to require: for example, algebraic 
expressions and repetitive loop structures. In program- 
ming for Artificial Intelligence we need to express very 
easily such instructions: 

To achieve Goal A, set up 

sub-goal B and if this 

fails, try C or D. 
PLANNER 1s a language designed to be highly expressive 
1n talking about just this kind of advice. 



Example : An example that illustrates PLANNER'S expres- 
sive power is that a graduate student, Ira Goldstein, 
was able to write a PLANNER program rather like the 
old Gelernter Geometrical -Theorem-Prover -- but not 
only did Goldstein need only a few days of work; once 
made, his program could easily be modified to handle 
classes of proof Involving constructions which Gelernter 
was unable to do in a long period of work. This 
ability to perform experiments 1n Artificial Intelligence 
rapidly and flexibly 1s a most decisive change in the 
style and fertility of work. 



(b) Automatic Mechanisms for Fallible and Contingent Instructions 

The image of a classical numerical program is as a sequence 
of actions performed one after the other and expected to 
succeed. In Artificial Intelligence, programming an action 
might be an attempt that fails. In this case it 1s necessary 
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to ensure that when it gives up, the micro-world in 
which it works Is not left untidely cluttered- The 
PLANNER system contains powerful mechanisms to take 
care of such situations. 



Exairple : The examples Already cited of how the Winograd 
program hangles its Blocks Words is a typical case. If 
the proposed action includes a statement-erasure the 
failure back-up can restore the description to its for- 
mer state with, perhaps, some editing relevant to the 
failure. 



(c) Procedural Description of Knowledge 

It is slightly Ironic that the most popular approaches 
to Artificial Intelligence programing force knowledge 
to be stated in a form of "logic" as assertive proposi- 
tions such as: 

The Box is Red, 
Now, there 1s no problem here, in simple statements 
about attributes. But other kinds of knowledge are 
much better stated as procedures (or "programs") 
rather than as facts , Even such a simple statement 

as: 

"If there are no cars coming, cross the road." 
is misleading If translated into a logical implication 
such as: 

For every x* [(x is car) & (x Is not coming)] 
- y [crossing is permissible]. 
It is more naturally transcribed as: 
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look left, look right. If you haven't seen 
a car cross - H 
The distinction between these modes of expression is not 
merely verbal. A deeper aspect is seen by picturing a 
"logical theorem prover" trying to prove by resolution 
or other logical principles that no car 1s coming I 
Deeper yet are the consequences in more complex situations 
we believe that even in the size of programs we are now 
using they make a difference between easy programming and 
very difficult programing. With more complexity we 
believe that the difference can become one of possible 
versus impossible. 

(d) An Example of Procedural Necessity 

The concept of "nearness" is a good example showing 
procedural definitions to represent knowledge. Everyone 
knows what "near" means. If we are told that 

The car is near the garage 
and 

The garage is near the house 
then we can be sure that the car Is near the house. But 
we cannot put this "transitivity" into a formal logical 
system by a rule like 
Rule 1 : (A near B) and (B near C)^ (A near C) 

for unrestricted application of this rule would yield 
absurdities like "1 is near 100" because 1 is near 
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1.001 and 1.001 1s near 1,002, etc., etc.! 

It is clear what is needed. In any particular 
context, "NEAR" is used to represent a certain size 
range, and if one uses chains of longer than a very 
few steps, one may get out of that range. The obvious 
thing to do is to add 

Rule 2 : Don't use Rule 1 more than (say) four times, 
unless tiiere is some basis for believing that you 
are still 1n the same size range. 

This cannot be said In any ordinary "logical system"! 
No system of "mathematical logic" allows statements 
Inside the logic to taU Jbout the deductive orocess 
that uses the logic. This, we claim, is disastrous 
for Intelligent systems because in solving a hard problem 
one must devote much attention to monitoring and 
planning the problem-solving activity! We can state 
Rule 2, or the equivalent, in the PLAffflER language. 
(The only earlier system we know of 1n which one could 
do this was Teitelman's PILOT.) 

Meta-techn1cal Remark 

This attitude is not shared by most other groups 
working on artificial Intelligence, and we feel that 
the widespread committment to try to represent ordinary 
reasoning 1n terns of a "consistent" mathematical logic 
system is having almost as bad an effect as was the 
earlier preoccupation with perceptrons and linear- 
separation clustering algorithms — another kind of 
attempt to find a uniform way to represent all different 
kinds of things. 

An Interesting sidelight on this is the phenomenon 
in which many people interpret GodeVs theorem as 
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showing a difference between men and machines. 
It does not. What 1t says is that any system which 
1s able to discuss Us own procedures, and apply 
these to Itself, has the potentiality of deducing 
some falsehoods. It Is perfectly possible to 
program computers to be able to discuss their own 
procedures — Winograd's program comes close to 
this — and we believe that this 1s the best 
path toward Intelligent programs; "Rule 2" 1s a 
simple Instance. We do not believe that enough 1s 
known, today, to make worthwhile the search for 
an adequate and "consistent" logical intelligent 
system (that 1s, one that is Inherently unable 
to have an^ self-contradictions). This has never 
been done even for ordinary mathematical arguments, 
to say nothing of everyday common sense. 

What men do 1s much more PlANNER-like. 
Suppose that you deduce a contradiction in some 
argwent or find that some plan you expected to 
work out did not. One then looks backwards and 
tries to "localize" the trouble. Then one makes 
a pattern-Invoked heuristic rule, to prevent that 
kind of deduction being made again. "Rule 2" 1s 
just such a device. A superb non-tr1v1al example 
1s provided by the w&y real mathematicians deal 
with "naive set theory". They do not reject it, 
as did Sertrand Russell in his attempt to rebuild 
mathematics without it. Instead, thty now have 
nrtes like: if your statement resembles Russell's 
Paradox (because it talks about Itself) then be 
careful; find another way to do it. It is astonish- 
ing how few such caveats have been adequate to 
keep contradictions out of ordinary mathematics. 
And we believe that among the most important forms 
of human knowledge are just such rules that indi- 
cate which lines of thinking are unsound. Another 
example: whenever one finds a theoretical outlook 
which can explain things "too easily", as do 
mystical concepts of "unity 1n everything" or the 
dialectical elements in Freud's Theory (in which 
many causes can produce opposite effects) one says 
to himself (unless he is still adolescent): This 
method Is too good. That means the inconsistency 
is too close to the purposes I want ft for." 
Then one tries to build up protective knowledge 
structures for preserving what one can (Freud 
proposes many valuable new Ideas about how know- 
ledge ts represented — and misrepresented!) of 
the new theory. 

In taking this path (and we believe it 1s 
the only promising approach, today, to under- 
standing Intelligence), we must be very clear 
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about the risks. We must understand that as our pro* 
grams get better at analyzing their own processes, and 
Incorporate better heuristics for preventing unsound 
kinds of reasoning from emerging, the possibilities 
of contradictions become burled more deeply but are 
not eliminated. We must not be entirely diverted by 
good empirical results. We must not give up on trying 
to get a complete theory. It is certainly not Incon- 
ceivable that we can construct a systematic theory of 
consistency In a logical area wide enough to cover 
all Important areas of intelligence; one must be on 
guard against other unwarranted {but ccmnon-sense!} 
pessimistic misinterpretations of Godel-lUe theories. 



(e) Mini-theory Construction as a Technical Goal^ 

We do not see the problem of Artificial Intelligence as 
one of programming existing knowledge ; it also Involves 
the acquisition and classification of new substantive 
knowledge about such areas as: intentions; excuses; 
goal structures end so on. A curious feature 1s that 
such enquiry Insofar as It has been conducted at all 
in the past, has been the domain of analytical philosophers 
and literary critics. Their analyses were, however, 
limited by their lack of computational models, so we 
are able to go further. Nevertheless, doing so depends 
on the acquisition, by people working on Artificial 
Intelligence, of sophistication in areas of thinking 
far removed from the content of "computer science" courses. 

Example : To illustrate the point we give an example from 
E. Cnarniak's work on making programs understand narra- 
tive. This work was mentioned in last year's proposal, 
and has recently reached the level of an operational 
program capable of answering questions such as in the 
following example. In the left column 1s a narrative 



taken from a children's reader. (In Its present form 
the program interacts 1n a special format; we have 
translated this into English for simplicity of reading.) 



STORY 



Jack and Janet are In the 
house. Jack Is holding a box 
of pencils and a box of paints. 

"Janet, see the paints and 
pencils that Daddy got for us," 
Jack said. 

Janet went to look at them* 

"Are the paints for me?" 
she asked. 

"No, the paints are mine/' 
said Jack. "The pencils are 
for yo u, Janet. " 

Janet said to herself/ I 
want the paints." 

Jack began to paint a pic- 
ture of a read airplane. Janet 
went to look at 1t. 

"Those paints make your air- 
plane look funny," she said. 
"You could pake a good picture 
of a red airplane with these 
pencils. 11 



Question: Why did Janet say that 
the paints were bad? 

Answer: She wants the paints. 

Comments: The program must Interpret 
"funny" as "bad" In this 
context (This 1s not done 
for ft In the Input format). 
Even then Janet really said 
that the picture was bad, 
and 1t is necessary to 
transfer this to the paints, 
and then 1t must know that 
if you want something another 
person has, you night make 
nasty conments about it in 
order to get 1t. If we had 
asked t Is the picture funny, 
the response would have been 
(in essence) "No, she said 
so but she had an ulterior 
motive." To do this the 
program needs a lot of infor- 
mation about wanting, trading, 
giving, owning. Statements can 
not be taken at face value -- 
translated into simple logical 
statements. They must be 
treated as evidence for the 
program to use to build a 
rodel of what really might 
be happening. 
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Specific Directions for 1972 
(a) Vision 

In last year's proposal the main goal for vision was creating 
a new heterarchlcal system 1n PLANNER, This was done by a team con* 
sistlng of P. Winston, B, Horn and E. Freuder. The new system 
quickly proved itself by performing a "copy demonstration" in which 
assemblies of blocks were analyzed, taken apart and re-assembled 1n 
mirror Image! This represents a very definite advance on the state 
of this art. 

This demonstration 1s significantly different from 
any we have made before in Its generality. The 
earlier demonstrations illustrated techniques that 
we could (and often later did) use in other contexts. 
But the demonstration Itself was usually rather 
rigidly unchangeable. The new vision system can be 
interfaced as it s ta n d s to almost any task • * . 
building towers, putting blocks into boxes etc. etc. 
Of course 1t has limitations and needs improvement. 
But it does not need to be reprogranmed to be used 
for a very general class of other tasks. 

The weakness of the new vision system is its failure to make 
effective use of all the heterarchlcal capabilities built into 1t. 
Indeed at the moment 1t works as if 1t were a series of separate 
heterarchlcal programs. But since the facilities for greater inter- 
action are there, 1t will make continuous progress. 

The most 1«med1ate weakness of this sort 1s a bottleneck 
of coonunl cation between finding lines and Interpreting 
scenes. Work 1s currently focussed on removing this. 

During the next year the system will be extended in the following 
directions: 
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(a) A series of new "mini-theories" to enable the 
system to use (rather than be bothered by!) 
shadow lines as a source of information* 

(b) More interactive line-finding* 

(c) Other representations of objects than as sets of 

lines. 

The possibility of multiple forms 
of representation for the basic 
elements of the system Is the 
consequence of heterarchy and one 
of the more exciting sources of 
theoretically interesting problems* 

(d) Range-finding. 

(e) The ability to confine its analysis to aspects of the 
scene with a high degree of relevance to the immediate 
question. At the moment the program still collects and 
analyzes much more Information than it needs* 

The following situation Illustrates whit we have in mind here* 





If the machine wants to pick up object A* it soon realizes that its 
position-locating module first requies some information about the 
size and position of 8. Consequently B is examined as a result of a 
goal to move A, but C is properly ignored as irrelevant to the goal. 
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(b) Robotics 

We shall concentrate on problems described in our separate 
letter on mini-robotics. 

(c) Language 

There are a number of new areas that demand Investigation: 

What are the problems in handling other 
aspects of ordinary gramnar? 

What new primitives will be needed as 
new words are added* etc. 

These problems are Inseparable from those that arise from adding a 

wider range of "meanings' 1 to the entire semantic system. What happens 

when we try to extend the system beyond those problems that it now 

can handle with respect to the blocks world? There are alternative 

concepts of how to proceed, and let us consider two extremes; 

One approach 1s to add to the blocks world incrementally. It 
is easy to add new kinds of objects, and properties for them, to 
the syntactic-semantic-problem solving complex* 

It is somewhat harder to add new predicates about spatial relations; 
for example, "NEXT-TO' might be important in some problems. But such 
an increment means that one must also add new procedures for taking 
account of such new elements, be they mentioned explicitly 1n a main 
goal derived from a natural language cocrcnand, or arise internally in a 
description Invoked by an already present theorem. The new procedures 
represent ways to solve problems and understand situations, but they 
cannot efficiently be used unless Veconmendations 11 are added to the 
theorems of the older knowledge base so that the new knowledge can be 
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Invoked when (and only when) relevant. Recommendations pointed the 
other way are needed, too, and modifications must be made to the 
Meanings" of appropriate words so that the syntactic and semantic systems 
can handle the nuances associated with the wider spectrum of meanings that 
the system is now required to deal with. 

Presumably, as this kind of incremental extension is made, some 
changes will be easy ~ whenever the system does something that "it 
should know better than to do" the programmer can intervene and attempt 
to adjoin new "advice" as a new theorem, recommendation, or entry in the 
dictionary, or fragment of PR0GRAW1AR or PLANNER program. But sometimes 
this will be found very difficult, because instead of a small addition 
or change the system will want a new kind of data-structure, or a new 
heuristic strategy for achieving a new kind of goal. 

For example, Winograd's Blocks World does not have a variety of 
"uses" for the mechanical structures it builds. In particular, it does 
not have any concept, at present, of multiple-support-upwards. It can 
deal with situations in which one object supports many others, as several 
stacks on a cube on a table, but it cannot handle such structures as 
bridges and arches tn which one object is supported by several* When 
a child builds a high tower that turns out to be unstable, he has ideas 
about "reinforcing" it by providing multiple support to lower elements. 

How hard is It to add such concepts? Me do not really know, yet. 
But 1t seems clear that it would be hard to do "incrementally" because 
the data-structure in the original blocks world assumed single support, 
and all the "theorems" about construction and interference between goals 
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are written in terms of this single support* 

Another approach opposed to the incremental would be based on a 
concept of "microworlds" (referring to suborganizations of knowledge, not 
to physically separable parts). The original "blocks World" is an elegant 
mini-theory which, by itself, is a highly satisfactory model of certain 
kinds of interactions between purposes and physical relations. Is there 
no way to preserve its effectiveness, intact. In a larger system? 

If the new an^ of meaning were very different it would be much 
more clear what to do. If, for example, we wanted to talk about what the 
objects of the blocks world were used for, (boxes are used to store things 
away that one does not expect to need soon, towers are used to make high 
structures, pyramids aren't used for anything, etc.) we would have little 
trouble. He could build a different micro-world about block-structures 
and their uses, and procedures for designing structures we needed for 
different purposes, and then turn to the blocks world to find how to make 
the structures. Presumably, it would not be excessively hard to add to 
the system a collection of theorems and reconrnendations that would serve 
to tie the two separate micro-worlds into a system that could solve 
problems that need both kinds of knowledge. 

Even this has not yet been done, however. So one of our goals is 
to accumulate experience in finding ways to interconnect two comparatively 
separate micro-worlds, another is to get experience in the kinds of 
problems involved in extending an existing micro-world. The first 
experience 1s, we predict, somewhat more valuable, for if it can properly 
be done, the interaction advice may be able to survive the extension of the 
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sub-microworlds involved. 

In any case, It 1s our conviction that there is no plausible 
alternative to this idea of structuring knowledge Into reasonably coherent 
packages . 

(d) PLAHHER 

Work is in progress on further and better implementation of 
the language. The limited version known as "micro-planner** 
is being used in several other centers. We like this, of 
course, but feel concerned lest a restricted form of the 
language become too well established, and will take what 
steps we can to make the best possible form generally 
available. In particular, we are concerned to make the 

language available over the ARPA net. 

(e) Program-Understanding Programs 

This 1s an area in which rapid growth Is very likely. It 
is closely related to the Natural Language project and to 
the structure of PUNNER. The explicit goal is to write 
programs capable of understanding programs. 
In addition to the central issues described before, we hope 
over the next year to go further into systems that understand, in various 
senses, more about processes. There are a number of different aspects 
of this general goal that arise over and over again in work on artificial 
Intelligence and, indeed, in many branches of computer science. For 
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example: 

Compilers do not have much Idea of what 
they are doing. Programs are compiled 
without any sense of the intention of 
the source program, and the resulting 
code is produced without any awareness 
of implications of the specified 
process. 

Operating systems do not know what they 
are doing, either. They are dealing 
directly with the execution of processes 
but have no semantic model of what is 
Involved; 

Learning programs like that of Winston produce descriptions of 
structures from a sequence of examples that have been presented. Much 
of learning involves the acquisition of new processes. He do not be- 
lieve that there is necessarily a large difference between acquiring 
descriptions that represent the structures of objects and descriptions 
that represent the specification of procedures. But if we are to be able 
to do the latter, we experience with probleiv-solvlng systems that deal 
directly with the semantics of programs, and this experience is generally 
lacking. 

More generally, in order for a learning program to be versatile* 
It has to be able to analyze procedures that it learns, to adapt them 
to new situations, to debug them, etc* We believe that human children 
who are not able to cope with complicated situations are that way because 
their process-understanding capability is inadequate. 

We are considering several approaches to developing competence in 
this area of procedure understanding programs. 
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1. Carl Hewitt has been developing a formalism, called INTENDER, 
In which one can associate with definitions of procedures semantic state- 
ments that make assertions about the effects that the procedural state- 
ments are supposed to have. In a sense, this Is a sort of formalization 
of the semantic clues programmers frequently leave In the form of 
mnemonics and comments* There are a variety of ways to use these statements, 
ranging from proving that the procedures will have the intended effects 

to making the procedures adaptable to use by larger systems that can 
understand and change the code, or translate It Into other source languages. 

2. Hewitt has also developed a system for "Procedural Abstraction, 11 
described 1n his thesis which observes partial protocols of the behavior 

of a program* The protocols provide evidence about the different directions 
that program branches can take, with different data, and the system con- 
structs proposed programs that are minimal models 1n the sense that they 
contain just enough structure to account for the behavior. In effect, 
the system is an abstract learning program that can duplicate behavior that 
it observes, creating as small a program as it can find (in the sense of 
doing as much as possible by means of loops). This 1s the kind of process 
that would be appropriate, for example, in a system to learn the grammar 
embodied in some program, by discovering that certain groups of words 
all have the same effect on the program's branching and goes on to 
discover what certain kinds of phrases all have similar roles in causing 
that branching. 

3. Ira Goldstein is developing a program capable of understanding 
very simple programs such as might be written by a beginning student. 



-27- 



A typical kind of task that this program will perform In Its elementary 
stages is to recognize the partial equivalence of the two LOGO programs 
PI and P2, the verbal descriptions P3 and P4 and a rough drawing of a 
square: 

PI TO XX 

1 FORWARD 100 

2 RIGHT 90 

3 FORWARD 100 

4 RIGHT 90 

5 FORWARD 100 

6 RIGHT 90 

7 FORWARD 100 

P2 TO 77 

1 FORWARD 100 

2 RIGHT 90 

3 77 

P3 Draw a line* then a right angle, then another line 

the same length, a right angle, another line, 
another right angle, another line. 

P4 Draw a line, turn a right angle and keep repeating. 
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4. Me have an extensive research program, funded by the N5F, to Inves- 
tigate how children describe, develop and think about their own 
processes. 

(f) Mathematics and Hath lab 

In cooperation with the groups in Project MAC, we plan to 
continue developing relevant background theories In schemata, 
complexity, and applied mathematics. 

(g) ITS and Programing 

We are very Interested in decentralizing our use of computers. 
Some ideas for doing this even for control of robotics are 
mentioned in the mini-robotics proposal. The situation is 
more straightforward in decentralizing programs of a purely 
symbolic kind and we will attempt to develop some of our new 
projects through network access to other computers provided that 
arranagements can be made to guarantee appropriate service. 

(h) Visitors 

Our faculty will be reinforced by long visits from M. Blum 
{Berkeley), Y. Shirai {Electrotechnical Laboratory, Japan) 
and possibly M. Rabin (Israel). 



APPENHIX 

Sample Dialog with Wlnogred's Language Understanding ^yate^ 

The Blocks World was designed by Terry Wlnograd to serve as a 
subject domain for the development of a language understanding system. 
This system is radical 1y more ambitious than earlier attempts to deal 
with natural language In several respects. It Is as much, or more, 
concerned with the oroblem of representing the meanings lr a discourse 
than with the grammatical structure; Wlnograd's thesis asserts that the 
Interactions between meanings and sysntactlc structures are so Intimate 
that It is a bad Idea to try to draw a definite boundary between the 
procedures that "parse 11 a sentence and the procedures that ^re to be used 
to drew logical Inferences about the meanings carried In th« discourse. 

WInograd's system Is a large. Intricate system. It Is described In his 
thesis at several levels of detail and It Is Impossible to do justice to 
It In any short summary. Our strategy will be first to present an 
example of Its performance In the blocks world context/ and then to 
discuss some of the mechanisms that are Involved In attaining this 
performance. In the following dialog the system carries out actions and 
answers questions about a simple scene containing a table, hand, and box, 
and several blocks and pyramids, as shown onnpige 0. The lines In lower 
case were typed In by a person, and the tines In upper case are the 
computer's response. Indented lines are consents describing come of the 

features being exhibited. 

The dialog was carried out In real time with the response of the 
"robot" displayed on a CRT screen, Figures are Included to show the 
state of the scene at various points, 

1, pick up a Die red block. 
....... OK. 

The system answers "OK" when It carries out a command. 
In order to pick up the red block. It had to clear It 
off by finding a space for the green one and moving 
the green one away. 

Insert Figure on p;ge 3 about here. 

2, grasp the pyramid. 

....... | DOtf'T UNDERSTAND WHICH PYRAMID YOU MEAN, 

There are three pyramids In the scene, so the program 
does nothing. 

3, find a Block which Is taller than the one you are holding and put It 
Into the box. 

The system has to figure out what Is meant by "one" 
and "It", as well as using deduction to decide which 
block It Is holding. 



..... BY "IT"/ I ASSUME YOU MEAN THE 3L0CK WHICH IS TALLER THAN THE ONE 
AM MOLDING, 



Printed out to keep us Informed. 
....... OK, 

The hand has to set down the red block to pick up th* 
taller blue one. Figure k shows the result of the 
system's manipulations so far. 

Insert Figure k about here. 

**, what does the box contain? 

- - THE BLUE PYRAMID AND THE BLUE BLOCK, 

5 . what Is the pyramid supported by? 

Earl 1 er the system responded to "the pyramid" by 
saying "I don't understand which pyramid you mean-" 
now. It assumes that we mean the one just mentioned. 
— — « THE BOX. 

6. how many blocks are not In the box? 

Negatives such as "not" and "no" are handled whether 
they appear In clauses or noun groups (such as "no 
boxes"). 
« FOUR OF THEM. 

7. Is at least one of them narrower than the one which I told you to 
pick up? 

The phrase "at least one of them" must be analyzed to 
refer back to the previous question, and "the one 
which.. ," must also refer to blocks. Noun croups can 
have complicated numbers such as "at least one", "more 
than three", "exactly seven", etc. The memory of past 
events Is used to deduce which block Is M the one which 
I told you to pick up 11 . Notice the complicated 
grammatical structure In which "the one" Is actually 
the object of a clause embedded two levels deeper. 
....... YES, THE RED CUBE. 

8. lilt supported? 

Passives can be used without a "by ..." phrase. 
....... YES, BY THE TABLE. 

9. can the table pick up blocks? 
....... NO, 

The semantic rules make this a nonsensical question 
which therefore must be false. 

10. can a pyramid be supported by a block? 
....... YES. 

The deductive system finds an actual example, so It 
knows this Is possible. 
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11. can a pyramid support a pyramid? 
...».» I DON'T KNOW. 

The question makes sense, but there Is no example In 
the scene, so there Is no easy way to tell without 
being able to examine the robot's programs. 

12. stack up two pyramids. 
....... [ CAN'T. 

The robot tries and falls. 

13. the blue pyramid Is mine. 
....... | UNDERSTAND. 

This data Is added to the data base. 

14. I own blocks which &re not red, but I don't own anythlnc which 
supports a pyramid. 

....... | UNDERSTAND. 

Two new theorems are created for proving things about 
"owning". 

15. do I own the box? 
....... no. 

The box supports whatever I t contains. Including a 

pyramid. 

16. do I own anything In the box? 

....... YES, TWO THINGS: THE BLUE BLOCK AND THE BLUE PYRAMID. 

17* will you please stack up both of the red blocks and either a green 
cub* or a pyramid? 
....... OK. 

Logical connectives such as "and 11 "or" "either" etc. 
are handled In both the grammar and semantics. A 
corrmand Is executed even If It looks like a question. 
The command Is carried out by the following series of 
steps: It puts a green cube on the large red block 
(note that It chooses the green cube with nothing on 
It), then removes the smal 1 pyramid from the little 
red cube, so that It can use that cube to complete the 
stack as specified, resulting In the situation of 
Figure 5. 

Insert Figure 5 about here. 

18. which cube Is sitting on the table? 

....... THE LARGE GREEN ONE WHICH SUPPORTS 

THE RED PYRAMID. 

Ordinarily the system would be satisfied with calling 
this "a big green cube", but since we asked "which 
cube" It has to be more specific In naming It. The 
use of substitute nouns (such as "one") and pronouns 
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In answers makes the discourse more natural, 

13. Is there a large block behind a pyramid? 
*•*».» YES, THREE OF THEM! A LARGE RED ONE, 

A LARGE GREEN CUBE AND THE BLUE ONE. 

20. put a small one onto the green cube which supports a pyramid. 

"A small one" must contrast with the earlier phrase "a 
large block", so this means "a small block"* 
....... OK. 

21. put the littlest pyramid on too of It. 

Words like "littlest" are not In the dictionary but 
art Interpreted from the root forms 1 1 ke "little"* 
....... OK. 

22. how many things are on top of green cubes? 

....... |>m NOT SURE WHAT YOU MEAN BY "ON TOP OF" IN THE PHRASE "ON TOP 

OF GREEN CUBES ". 
00 YOU MEAN: 

- 1 - DIRECTLY ON THE SURFACE 
2 - ANWHERE ON TOP OF ? 

When there %r€ several Interpretations and none Is 
clearly better, the system has to ask the user for 
mora Information. The paraphrases were Included with 
the dictionary definition of "on", and the request for 
more Information occurs automatical ly when the 
heurlstl cs for resolving ambiguities aren't 

sufficient. A request like this can also occur when 
the meaning of a pronoun such as "It" or "they" Is 
unclear. 
2 ...... THREE OF THEM. 

25. had you touched any pyramid before you put the green one on the 

little cube? 

« YES, THE GREEN ONE. 

The system deduces from Its memory what event Is being 
referred to, and can relate other events to it In 
time. The anal ysls Includes complex tenses of verb 

groups, and there are special facilities for kpeplng 
track of the various forms of Irregular verbs like 
"have". 

2i*. when did you pick It up? 

....... WHILE I WAS STACKING UP THE RED CUBE, A LARGE RED BLOCK AND A 

LARGE GREEN CUBE. 

A time Is described by naming the main goal which was 

being worked on. 
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25. why? 

The system remembers Immediately previous sentences to 
understand questions like this and the following ones. 
....... TO GET RID OF IT. 

By keeping track of selected parts of the original 
subgoal tree/ the system has some understanding of Its 
own motives. 

26. why did you do that? 
....... TO CLEAN OFF THE RED CUBE. 

"That" refers to the action mentioned In the answer. 

27. why did you clear off that cube? 

Here/ "that" means the one just referred to. 
....... TO PUT IT ON A LARGE GREEN CUBE. 

28. why did you do that? 

....... TO STACK UP THE RED CUBE , A LARGE RED BLOCK AND A LARGE GREEN 

CUBE. 

29* why did you do that? 

.... ... BECAUSE YOU ASKED HE TO. 

It remembers what It did, not how the request was 

w&fded. 

30. how did you do It? 

"It" refers to the action mentioned In the last 

question, rather than the answer. 
....... ay PUTTING A LARGE RED BLOCK ON THE TABLE j THEN LETTING GO OF 

IT ; THEN PUTTING A LARGE GREEN CUBE ON IT j THEN LETTING GO OF THAT 
CUBE ; THEN PUTTING THE RED CUBE ON THAT CUBE ; THEN LETTING GO OF THAT 
CUBE. 

This Is a description of the top-level subgoal 5 used 

to achieve the goal * 

51. how many objects did you touch while you were doing It? 

Time references can refer to a time span, using words 

like "while". 
FOUR OF THEM. 

32. what did the red cube support before you started to clean It off? 
....... THE GREEN PYRAMID. 

By remembering what has been moved. It Is possible to 

deduce things about earlier states of the scene. We 

can use complex nestings of clauses like "before you 
started to clean It off". 

33, there were five blocks to the left of the box then. 

"Then" must refer to the time mentioned In the last 
sentence. 
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....... no # ONLY FOUR OF THEM: THE RED CUBE, TWO LARGE GREEN CUBES AND A 

LARGE RED BLOCK. 

Earlier In the dialog. Information about M ownine M was 
accepted at face value. But the system knows that It 
has complete Inf ormat Ion about about the basic 
properties of objects. Including location. It checks 
any statements about location or other such properties 
to see whether It agrees or disagrees. 

3ii, put the blue pyramid on the block in the box. 

This Is syntactically ambiguous, but Is understood 
without ambiguity since the parsing corresponding to 
"put the blue pyramid which Is on the block In the 
box.' 1 does not make sense In the current scene, while 
"put the blue pyramid on the block which Is In the 
box" does. 

«"«- OK. 

35* Is there anything which Is bigger than every pyramid but Is not as 

wide as the thing that supports It? 

The system understands quantifiers such as "every", 
"any", "some", etc., and structures such as "bigger 
than", "as wide as", etc. Here "It" refers to a noun 
group within the same sentence but two levels up In 
the parsing tree, 

....... yes, THE BLUE BLOCK. 

36. does a steeple 

SORRY, I DON'T KNOW THE WORD "STEEPLE". 

The system Interrupts Immediately when It sees a word 
I t can' t figure out. 

37. a "steeple" Is a stack which contains two green cubes and a pyramid. 

The quote marks Indicate that we are defining a new 
word. 
....... 1 UNDERSTAND. 

38. ere there mny steeples now? 
NO. 

The new definition Is used as a description to 
recognize steeples. 

39. bul Id one. 
— --« OK. 

The description given In the definition Is enough to 
deduce a plan for building new objects as well as 
recognizing them. The word "one" means "a steeple", 
and the plan Is carried out. First, one of the large 
green cubes Is cleared off by removing the red cube 
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and two pyramids. Then the two green cubes and the 
tall red pyramid are stacked up to fit the 
description. 



UO. call the biggest block n superblock". 
OK. 

We can also give objects names. 



■ 



ftl* have you picked up suoerblock since we began? 
«««- YES. 

i*2. why did you drop It? 

....... BY "IT", I ASSUME YOU MEAN THE SUPER8L0CK. 

....... | CAN'T EXPLAIN A NON-EXISTENT EVENT. 

There Is no record In memory of any block being 

dropped* 

43. Is there anything to the right of the red pyramid? 

....... YES, FOUR THINGS: THE BOX , THE RED CUBE, THE BLUE PYRAMID AND 

THE BLOCK NAMED SUPERBLOCK 

Once an object has a name, the name Is used to help 

I dent I f y It In responses. 

-bit. thank you. 

...... YOU'RE WELCOMEI 

Idioms can be defined without modifying the grammar. 
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Figure 3 
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or 




1*7. thank you. 
YOU ! RE WELCOME! 



The special Idlom-def Inlng capabilities allow 
us to Include such pleasantries without 
modi fy Ing the grammar.' 



